Motion prediction in video coding

ABSTRACT

There is disclosed apparatuses, methods and computer programs for utilizing motion prediction in video coding. A block of pixels of a video representation encoded in a bitstream is read, and a type of the block is determined. If the determining indicates that the block is a block predicted by using two or more reference blocks, a first reference pixel location in a first reference block is determined and a second reference pixel location in a second reference block is determined. The first reference pixel location is used to obtain a first prediction. Said first prediction has a second precision, which is higher than the first precision. The second reference pixel location is used to obtain a second prediction, which also has the second precision. The first prediction and the second prediction are combined to obtain a combined prediction; and the precision of the combined prediction is reduced to the first precision.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of and claims priority to U.S.application Ser. No. 13/344,893, filed on Jan. 6, 2012, which is anon-provisional that claims the benefit of U.S. Provisional ApplicationNo. 61/430,694, filed Jan. 7, 2011, all of the contents of each of whichare hereby incorporated in their entireties.

TECHNICAL FIELD

The present invention relates to an apparatus, a method and a computerprogram for producing and utilizing motion prediction information invideo encoding and decoding.

BACKGROUND INFORMATION

A video codec may comprise an encoder which transforms input video intoa compressed representation suitable for storage and/or transmission anda decoder that can uncompress the compressed video representation backinto a viewable form, or either one of them. The encoder may discardsome information in the original video sequence in order to representthe video in a more compact form, for example at a lower bit rate.

Many hybrid video codecs, operating for example according to theInternational Telecommunication Union's ITU-T H.263 and H.264 codingstandards, encode video information in two phases. In the first phase,pixel values in a certain picture area or “block” are predicted. Thesepixel values can be predicted, for example, by motion compensationmechanisms, which involve finding and indicating an area in one of thepreviously encoded video frames (or a later coded video frame) thatcorresponds closely to the block being coded. Additionally, pixel valuescan be predicted by spatial mechanisms which involve finding andindicating a spatial region relationship, for example by using pixelvalues around the block to be coded in a specified manner.

Prediction approaches using image information from a previous (or alater) image can also be called as Inter prediction methods, andprediction approaches using image information within the same image canalso be called as Intra prediction methods.

The second phase is one of coding the error between the predicted blockof pixels and the original block of pixels. This is typicallyaccomplished by transforming the difference in pixel values using aspecified transform. This transform may be e.g. a Discrete CosineTransform (DCT) or a variant thereof. After transforming the difference,the transformed difference may be quantized and entropy encoded.

By varying the fidelity of the quantization process, the encoder cancontrol the balance between the accuracy of the pixel representation,(in other words, the quality of the picture) and the size of theresulting encoded video representation (in other words, the file size ortransmission bit rate).

An example of the encoding process is illustrated in FIG. 1.

The decoder reconstructs the output video by applying a predictionmechanism similar to that used by the encoder in order to form apredicted representation of the pixel blocks (using the motion orspatial information created by the encoder and stored in the compressedrepresentation of the image) and prediction error decoding (the inverseoperation of the prediction error coding to recover the quantizedprediction error signal in the spatial domain).

After applying pixel prediction and error decoding processes the decodercombines the prediction and the prediction error signals (the pixelvalues) to form the output video frame.

The decoder (and encoder) may also apply additional filtering processesin order to improve the quality of the output video before passing itfor display and/or storing as a prediction reference for the forthcomingframes in the video sequence.

An example of the decoding process is illustrated in FIG. 2.

Motion Compensated Prediction (MCP) is a technique used by videocompression standards to reduce the size of an encoded bitstream. InMCP, a prediction for a current frame is formed using a previously codedframe(s), where only the difference between original and predictionsignals, representative of the current and predicted frames, is encodedand sent to a decoder. A prediction signal, representative of aprediction frame, is formed by first dividing a current frame intoblocks, e.g., macroblocks, and searching for a best match in a referenceframe for each block. In this way, the motion of a block relative to thereference frame is determined and this motion information is coded intoa bitstream as motion vectors. A decoder is able to reconstruct theexact prediction frame by decoding the motion vector data encoded in thebitstream.

An example of a prediction structure is presented in FIG. 8. Boxesindicate pictures, capital letters within boxes indicate coding types,numbers within boxes are picture numbers (in decoding order), and arrowsindicate prediction dependencies. In this example I-pictures are intrapictures which do not use any reference pictures and thus can be decodedirrespective of the decoding of other pictures. P-pictures are so calleduni-predicted pictures i.e. they refer to one reference picture, andB-pictures are bi-predicted pictures which use two other pictures asreference pictures, or two prediction blocks within one referencepicture. In other words, the reference blocks relating to the B-picturemay be in the same reference picture (as illustrated with the two arrowsfrom picture P7 to picture B8 in FIG. 8) or in two different referencepictures (as illustrated e.g. with the arrows from picture P2 and frompicture B3 to picture B4 in FIG. 8).

It should also be noted here that one picture may include differenttypes of blocks i.e. blocks of a picture may be intra-blocks,uni-predicted blocks, and/or bi-predicted blocks. Motion vectors oftenrelate to blocks wherein for one picture a plurality of motion vectorsmay exist.

In some systems the uni-predicted pictures are also called asuni-directionally predicted pictures and the bi-predicted pictures arecalled as bi-directionally predicted pictures.

The motion vectors are not limited to having full-pixel accuracy, butcould have fractional-pixel accuracy as well. That is, motion vectorscan point to fractional-pixel positions/locations of the referenceframe, where the fractional-pixel locations can refer to, for example,locations “in between” image pixels. In order to obtain samples atfractional-pixel locations, interpolation filters may be used in the MCPprocess. Conventional video coding standards describe how a decoder canobtain samples at fractional-pixel accuracy by defining an interpolationfilter. In MPEG-2, for example, motion vectors can have at most,half-pixel accuracy, where the samples at half-pixel locations areobtained by a simple averaging of neighboring samples at full-pixellocations. The H.264/AVC video coding standard supports motion vectorswith up to quarter-pixel accuracy. Furthermore, in the H.264/AVC videocoding standard, half-pixel samples are obtained through the use ofsymmetric and separable 6-tap filters, while quarter-pixel samples areobtained by averaging the nearest half or full-pixel samples.

In typical video codecs, the motion information is indicated by motionvectors associated with each motion compensated image block. Each ofthese motion vectors represents the displacement of the image block inthe picture to be coded (in the encoder) or decoded (at the decoder) andthe prediction source block in one of the previously coded or decodedimages (or pictures). In order to represent motion vectors efficiently,motion vectors are typically coded differentially with respect to blockspecific predicted motion vector. In a typical video codec, thepredicted motion vectors are created in a predefined way, for example bycalculating the median of the encoded or decoded motion vectors of theadjacent blocks.

In typical video codecs the prediction residual after motioncompensation is first transformed with a transform kernel (like DCT) andthen coded. The reason for this is that often there still exists somecorrelation among the residual and transform can in many cases helpreduce this correlation and provide more efficient coding.

Typical video encoders utilize the Lagrangian cost function to findoptimal coding modes, for example the desired macro block mode andassociated motion vectors. This type of cost function uses a weightingfactor or λ to tie together the exact or estimated image distortion dueto lossy coding methods and the exact or estimated amount of informationrequired to represent the pixel values in an image area.

This may be represented by the equation:C=D+□λR  (1)

where C is the Lagrangian cost to be minimised, D is the imagedistortion (for example, the mean-squared error between the pixel valuesin original image block and in coded image block) with the mode andmotion vectors currently considered, λ is a Lagrangian coefficient and Ris the number of bits needed to represent the required data toreconstruct the image block in the decoder (including the amount of datato represent the candidate motion vectors).

Some hybrid video codecs, such as H.264/AVC, utilize bi-directionalmotion compensated prediction to improve the coding efficiency. Inbi-directional prediction, prediction signal of the block may be formedby combining, for example by averaging two motion compensated predictionblocks. This averaging operation may further include either up or downrounding, which may introduce rounding errors.

The accumulation of rounding errors in bi-directional prediction maycause degradation in coding efficiency. This rounding error accumulationmay be removed or decreased by signalling whether rounding up orrounding down have been used when the two prediction signals have beencombined for each frame. Alternatively the rounding error could becontrolled by alternating the usage of the rounding up and rounding downfor each frame. For example, rounding up may be used for every otherframe and, correspondingly, rounding down may be used for every otherframe.

In FIG. 9 an example of averaging two motion compensated predictionblocks using rounding is illustrated. Sample values of the firstprediction reference is input 902 to a first filter 904 in which valuesof two or more full pixels near the point which the motion vector isreferring to are used in the filtering. A rounding offset may be added906 to the filtered value. The filtered value added with the roundingoffset is right shifted 908 x-bits i.e. divided by 2^(x) to obtain afirst prediction signal P1. Similar operation is performed to the secondprediction reference as is illustrated with blocks 912, 914, 916 and 918to obtain a second prediction signal P2. The first prediction signal P1and the second prediction signal P2 are combined e.g. by summing theprediction signals P1, P2. A rounding offset may be added 920 with thecombined signal after which the result is right shifted y-bits i.e.divided by 2^(y). The rounding may be upwards, if the rounding offset ispositive, or downwards, if the rounding offset is negative. Thedirection of the rounding may always be the same, or it may alter fromtime to time, e.g. for each frame. The direction of the rounding may besignaled in the bitstream so that in the decoding process the samerounding direction can be used.

However, these methods increase somewhat the complexity as two separatecode branches need to be written for bi-directional averaging. Inaddition, the motion estimation routines in the encoder may need to bedoubled for both cases of rounding and truncation.

SUMMARY

The present invention introduces a method which enables reducing theeffect of rounding errors in bi-directional and multi-directionalprediction. According to some embodiments of the invention predictionsignals are maintained in a higher precision during the predictioncalculation and the precision is reduced after the two or moreprediction signals have been combined with each other.

In some example embodiments prediction signals are maintained in higheraccuracy until the prediction signals have been combined to obtain thebi-directional or multidirectional prediction signal. The accuracy ofthe bi-directional or multidirectional prediction signal can then bedownshifted to an appropriate accuracy for post processing purposes.Then, no rounding direction indicator need not be included in or readfrom the bitstream

According to a first aspect of the present invention there is provided amethod comprising:

determining a block of pixels of a video representation encoded in abitstream, values of said pixels having a first precision;

determining a type of the block;

if the determining indicates that the block is a block predicted byusing two or more reference blocks,

determining a first reference pixel location in a first reference blockand a second reference pixel location in a second reference block;

using said first reference pixel location to obtain a first prediction,said first prediction having a second precision, which is higher thansaid first precision;

using said second reference pixel location to obtain a secondprediction, said second prediction having the second precision, which ishigher than said first precision;

combining said first prediction and said second prediction to obtain acombined prediction; and

decreasing the precision of said combined prediction to said firstprecision.

According to a second aspect of the present invention there is providedan apparatus comprising:

a processor; and

a memory unit operatively connected to the processor and including:

computer code configured to determine a block of pixels of a videorepresentation encoded in a bitstream, values of said pixels having afirst precision;

computer code configured to determine a type of the block;

computer code configured to, if the determining indicates that the blockis a block predicted by using two or more reference blocks,

determine a first reference pixel location in a first reference blockand a second reference pixel location in a second reference block;

use said first reference pixel location to obtain a first prediction,said first prediction having a second precision, which is higher thansaid first precision;

use said second reference pixel location to obtain a second prediction,said second prediction having the second precision, which is higher thansaid first precision;

combine said first prediction and said second prediction to obtain acombined prediction; and

decrease the precision of said combined prediction to said firstprecision.

According to a third aspect of the present invention there is provided acomputer readable storage medium stored with code thereon for use by anapparatus, which when executed by a processor, causes the apparatus toperform:

determine a block of pixels of a video representation encoded in abitstream, values of said pixels having a first precision;

determine a type of the block;

if the determining indicates that the block is a block predicted byusing two or more reference blocks,

determine a first reference pixel location in a first reference blockand a second reference pixel location in a second reference block;

use said first reference pixel location to obtain a first prediction,said first prediction having a second precision, which is higher thansaid first precision;

use said second reference pixel location to obtain a second prediction,said second prediction having the second precision, which is higher thansaid first precision;

combine said first prediction and said second prediction to obtain acombined prediction; and

decrease the precision of said combined prediction to said firstprecision.

According to a fourth aspect of the present invention there is providedat least one processor and at least one memory, said at least one memorystored with code thereon, which when executed by said at least oneprocessor, causes an apparatus to perform:

determine a block of pixels of a video representation encoded in abitstream, values of said pixels having a first precision;

determine a type of the block;

if the determining indicates that the block is a block predicted byusing two or more reference blocks,

determine a first reference pixel location in a first reference blockand a second reference pixel location in a second reference block;

use said first reference pixel location to obtain a first prediction,said first prediction having a second precision, which is higher thansaid first precision;

use said second reference pixel location to obtain a second prediction,said second prediction having the second precision, which is higher thansaid first precision;

combine said first prediction and said second prediction to obtain acombined prediction; and

decrease the precision of said combined prediction to said firstprecision.

According to a fifth aspect of the present invention there is providedan apparatus comprising:

an input to determine a block of pixels of a video representationencoded in a bitstream, values of said pixels having a first precision;

a determinator to determine a type of the block; wherein if thedetermining indicates that the block is a block predicted by using twoor more reference blocks, said determinator further to determine a firstreference pixel location in a first reference block and a secondreference pixel location in a second reference block;

a first predictor to use said first reference pixel location to obtain afirst prediction, said first prediction having a second precision, whichis higher than said first precision;

a second predictor to use said second reference pixel location to obtaina second prediction, said second prediction having the second precision,which is higher than said first precision;

a combiner to combine said first prediction and said second predictionto obtain a combined prediction; and

a shifter to decrease the precision of said combined prediction to saidfirst precision.

According to a sixth aspect of the present invention there is providedan apparatus comprising:

means for determining a block of pixels of a video representationencoded in a bitstream, values of said pixels having a first precision;

means for determining a type of the block;

means for determining a first reference pixel location in a firstreference block and a second reference pixel location in a secondreference block, if the determining indicates that the block is a blockpredicted by using two or more reference blocks;

means for using said first reference pixel location to obtain a firstprediction, said first prediction having a second precision, which ishigher than said first precision;

means for using said second reference pixel location to obtain a secondprediction, said second prediction having the second precision, which ishigher than said first precision;

means for combining said first prediction and said second prediction toobtain a combined prediction; and

means for decreasing the precision of said combined prediction to saidfirst precision.

This invention removes the need to signal the rounding offset or usedifferent methods for rounding for different frames. This invention maykeep the motion compensated prediction signal of each one of thepredictions at highest precision possible after interpolation andperform the rounding to the bit-depth range of the video signal afterboth prediction signals are added.

DESCRIPTION OF THE DRAWINGS

For better understanding of the present invention, reference will now bemade by way of example to the accompanying drawings in which:

FIG. 1 shows schematically an electronic device employing someembodiments of the invention;

FIG. 2 shows schematically a user equipment suitable for employing someembodiments of the invention;

FIG. 3 further shows schematically electronic devices employingembodiments of the invention connected using wireless and wired networkconnections;

FIG. 4a shows schematically an embodiment of the invention asincorporated within an encoder;

FIG. 4b shows schematically an embodiment of an inter predictoraccording to some embodiments of the invention;

FIG. 5 shows a flow diagram showing the operation of an embodiment ofthe invention with respect to the encoder as shown in FIG. 4 a;

FIG. 6 shows a schematic diagram of a decoder according to someembodiments of the invention;

FIG. 7 shows a flow diagram of showing the operation of an embodiment ofthe invention with respect to the decoder shown in FIG. 6;

FIG. 8 illustrates an example of a prediction structure in a videosequence;

FIG. 9 depicts an example of a bit stream of an image;

FIG. 10 depicts an example of bi-directional prediction using rounding;

FIG. 11 depicts an example of bi-directional prediction according to anexample embodiment of the present invention; and

FIG. 12 illustrates an example of some possible prediction directionsfor a motion vector.

DETAILED DESCRIPTION OF SOME EXAMPLE EMBODIMENTS

The following describes in further detail suitable apparatus andpossible mechanisms for the provision of reducing information to betransmitted in video coding systems and more optimal codeword mappingsin some embodiments. In this regard reference is first made to FIG. 1which shows a schematic block diagram of an exemplary apparatus orelectronic device 50, which may incorporate a codec according to anembodiment of the invention.

The electronic device 50 may for example be a mobile terminal or userequipment of a wireless communication system. However, it would beappreciated that embodiments of the invention may be implemented withinany electronic device or apparatus which may require encoding anddecoding or encoding or decoding video images.

The apparatus 50 may comprise a housing 30 for incorporating andprotecting the device. The apparatus 50 further may comprise a display32 in the form of a liquid crystal display. In other embodiments of theinvention the display may be any suitable display technology suitable todisplay an image or video. The apparatus 50 may further comprise akeypad 34. In other embodiments of the invention any suitable data oruser interface mechanism may be employed. For example the user interfacemay be implemented as a virtual keyboard or data entry system as part ofa touch-sensitive display. The apparatus may comprise a microphone 36 orany suitable audio input which may be a digital or analogue signalinput. The apparatus 50 may further comprise an audio output devicewhich in embodiments of the invention may be any one of: an earpiece 38,speaker, or an analogue audio or digital audio output connection. Theapparatus 50 may also comprise a battery 40 (or in other embodiments ofthe invention the device may be powered by any suitable mobile energydevice such as solar cell, fuel cell or clockwork generator). Theapparatus may further comprise an infrared port 42 for short range lineof sight communication to other devices. In other embodiments theapparatus 50 may further comprise any suitable short range communicationsolution such as for example a Bluetooth wireless connection or aUSB/firewire wired connection.

The apparatus 50 may comprise a controller 56 or processor forcontrolling the apparatus 50. The controller 56 may be connected tomemory 58 which in embodiments of the invention may store both data inthe form of image and audio data and/or may also store instructions forimplementation on the controller 56. The controller 56 may further beconnected to codec circuitry 54 suitable for carrying out coding anddecoding of audio and/or video data or assisting in coding and decodingcarried out by the controller 56.

The apparatus 50 may further comprise a card reader 48 and a smart card46, for example a UICC and UICC reader for providing user informationand being suitable for providing authentication information forauthentication and authorization of the user at a network.

The apparatus 50 may comprise radio interface circuitry 52 connected tothe controller and suitable for generating wireless communicationsignals for example for communication with a cellular communicationsnetwork, a wireless communications system or a wireless local areanetwork. The apparatus 50 may further comprise an antenna 44 connectedto the radio interface circuitry 52 for transmitting radio frequencysignals generated at the radio interface circuitry 52 to otherapparatus(es) and for receiving radio frequency signals from otherapparatus(es).

In some embodiments of the invention, the apparatus 50 comprises acamera capable of recording or detecting individual frames which arethen passed to the codec 54 or controller for processing. In someembodiments of the invention, the apparatus may receive the video imagedata for processing from another device prior to transmission and/orstorage. In some embodiments of the invention, the apparatus 50 mayreceive either wirelessly or by a wired connection the image forcoding/decoding.

With respect to FIG. 3, an example of a system within which embodimentsof the present invention can be utilized is shown. The system 10comprises multiple communication devices which can communicate throughone or more networks. The system 10 may comprise any combination ofwired or wireless networks including, but not limited to a wirelesscellular telephone network (such as a GSM, UMTS, CDMA network etc), awireless local area network (WLAN) such as defined by any of the IEEE802.x standards, a Bluetooth personal area network, an Ethernet localarea network, a token ring local area network, a wide area network, andthe Internet.

The system 10 may include both wired and wireless communication devicesor apparatus 50 suitable for implementing embodiments of the invention.

For example, the system shown in FIG. 3 shows a mobile telephone network11 and a representation of the internet 28. Connectivity to the internet28 may include, but is not limited to, long range wireless connections,short range wireless connections, and various wired connectionsincluding, but not limited to, telephone lines, cable lines, powerlines, and similar communication pathways.

The example communication devices shown in the system 10 may include,but are not limited to, an electronic device or apparatus 50, acombination of a personal digital assistant (PDA) and a mobile telephone14, a PDA 16, an integrated messaging device (IMD) 18, a desktopcomputer 20, a notebook computer 22. The apparatus 50 may be stationaryor mobile when carried by an individual who is moving. The apparatus 50may also be located in a mode of transport including, but not limitedto, a car, a truck, a taxi, a bus, a train, a boat, an airplane, abicycle, a motorcycle or any similar suitable mode of transport.

Some or further apparatus may send and receive calls and messages andcommunicate with service providers through a wireless connection 25 to abase station 24. The base station 24 may be connected to a networkserver 26 that allows communication between the mobile telephone network11 and the internet 28. The system may include additional communicationdevices and communication devices of various types.

The communication devices may communicate using various transmissiontechnologies including, but not limited to, code division multipleaccess (CDMA), global systems for mobile communications (GSM), universalmobile telecommunications system (UMTS), time divisional multiple access(TDMA), frequency division multiple access (FDMA), transmission controlprotocol-internet protocol (TCP-IP), short messaging service (SMS),multimedia messaging service (MIMS), email, instant messaging service(IMS), Bluetooth, IEEE 802.11 and any similar wireless communicationtechnology. A communications device involved in implementing variousembodiments of the present invention may communicate using various mediaincluding, but not limited to, radio, infrared, laser, cableconnections, and any suitable connection.

Various embodiments can extend conventional two-stage sub-pixelinterpolation algorithms, such as the algorithm used in the H.264/AVCvideo coding standard, without the need to increase the complexity ofthe decoder. It should be noted here that FIG. 11 illustrates only somefull pixel values which are the nearest neighbors to the example blockof pixels but in the interpolation it may also be possible to use fullpixel values located farther from the block under consideration.Furthermore, the present invention is not only limited toimplementations using one-dimensional interpolation but the fractionalpixel samples can also be obtained using more complex interpolation orfiltering.

It should be noted that various embodiments can be implemented by and/orin conjunction with other video coding standards besides the H.264/AVCvideo coding standard.

With respect to FIG. 4a , a block diagram of a video encoder suitablefor carrying out embodiments of the invention is shown. Furthermore,with respect to FIG. 5, the operation of the encoder exemplifyingembodiments of the invention specifically with respect to theutilization of higher accuracy calculation of prediction signals isshown as a flow diagram.

FIG. 4a shows the encoder as comprising a pixel predictor 302,prediction error encoder 303 and prediction error decoder 304. FIG. 4aalso shows an embodiment of the pixel predictor 302 as comprising aninter-predictor 306, an intra-predictor 308, a mode selector 310, afilter 316, and a reference frame memory 318. The mode selector 310comprises a block processor 381 and a cost evaluator 382. FIG. 4b alsodepicts an embodiment of the inter-predictor 306 which comprises a blockselector 360 and a motion vector definer 361, which may be implementede.g. in a prediction processor 362. The inter-predictor 306 may alsohave access to a parameter memory 404. The mode selector 310 may alsocomprise a quantizer 384.

The pixel predictor 302 receives the image 300 to be encoded at both theinter-predictor 306 (which determines the difference between the imageand a motion compensated reference frame 318) and the intra-predictor308 (which determines a prediction for an image block based only on thealready processed parts of current frame or picture). The output of boththe inter-predictor and the intra-predictor are passed to the modeselector 310. The intra-predictor 308 may have more than oneintra-prediction modes. Hence, each mode may perform theintra-prediction and provide the predicted signal to the mode selector310. The mode selector 310 also receives a copy of the image 300.

The block processor 381 determines which encoding mode to use to encodethe current block. If the block processor 381 decides to use aninter-prediction mode it will pass the output of the inter-predictor 306to the output of the mode selector 310. If the block processor 381decides to use an intra-prediction mode it will pass the output of oneof the intra-predictor modes to the output of the mode selector 310.

According to some example embodiments the pixel predictor 302 operatesas follows. The inter predictor 306 and the intra prediction modes 308perform the prediction of the current block to obtain predicted pixelvalues of the current block. The inter predictor 306 and the intraprediction modes 308 may provide the predicted pixel values of thecurrent block to the block processor 381 for analyzing which predictionto select. In addition to the predicted values of the current block, theblock processor 381 may, in some embodiments, receive an indication of adirectional intra prediction mode from the intra prediction modes.

The block processor 381 examines whether to select the inter predictionmode or the intra prediction mode. The block processor 381 may use costfunctions such as the equation (1) or some other methods to analyzewhich encoding method gives the most efficient result with respect to acertain criterion or criteria. The selected criteria may include codingefficiency, processing costs and/or some other criteria. The blockprocessor 381 may examine the prediction for each directionality i.e.for each intra prediction mode and inter prediction mode and calculatethe cost value for each intra prediction mode and inter prediction mode,or the block processor 381 may examine only a subset of all availableprediction modes in the selection of the prediction mode.

In some embodiments the inter predictor 306 operates as follows. Theblock selector 360 receives a current block to be encoded (block 504 inFIG. 5) and examines whether a previously encoded image contains a blockwhich may be used as a reference to the current block (block 505). Ifsuch a block is found from the reference frame memory 318, the motionestimator 365 may determine whether the current block could be predictedby using one or two (or more) reference blocks i.e. whether the currentblock could be a uni-predicted block or a bi-predicted block (block506). If the motion estimator 365 has determined to use uni-prediction,the motion estimator 365 may indicate the reference block to the motionvector definer 361. If the motion estimator 365 has selected to usebi-prediction, the motion estimator 365 may indicate both referenceblocks, or if more than two reference blocks have been selected, all theselected reference blocks to the motion vector definer 361. The motionvector definer 361 utilizes the reference block information and definesa motion vector (block 507) to indicate the correspondence betweenpixels of the current block and the reference block(s).

In some embodiments the inter predictor 306 calculates a cost value forboth one-directional and bi-directional prediction and may then selectwhich kind of prediction to use with the current block.

In some embodiments the motion vector may point to a full pixel sampleor to a fraction pixel sample i.e. to a half pixel, to a quarter pixelor to a one-eighth pixel. The motion vector definer 361 may examine thetype of the current block to determine whether the block is abi-predicted block or another kind of a block (block 508). The type maybe determined by the block type indication 366 which may be provided bythe block selector 360 or another element of the encoder. If the type ofthe block is a bi-predicted block, two (or more) motion vectors aredefined by the motion vector definer 361 (block 509). Otherwise, if theblock is a uni-predicted block, one motion vector shall be defined(block 510).

It is also possible that the type of the block is determined before themotion vector is calculated.

The motion vector definer 361 provides motion vector information to theblock processor 381 which uses this information to obtain the predictionsignal.

When the cost has been calculated with respect to intra prediction modeand possibly with respect to the inter prediction mode(s), the blockprocessor 381 selects one intra prediction mode or the inter predictionmode for encoding the current block.

When the inter prediction mode was selected, the predicted pixel valuesor predicted pixel values quantized by the optional quantizer 384 areprovided as the output of the mode selector.

The output of the mode selector is passed to a first summing device 321.The first summing device may subtract the pixel predictor 302 outputfrom the image 300 to produce a first prediction error signal 320 whichis input to the prediction error encoder 303.

The pixel predictor 302 further receives from a preliminaryreconstructor 339 the combination of the prediction representation ofthe image block 312 and the output 338 of the prediction error decoder304. The preliminary reconstructed image 314 may be passed to theintra-predictor 308 and to a filter 316. The filter 316 receiving thepreliminary representation may filter the preliminary representation andoutput a final reconstructed image 340 which may be saved in a referenceframe memory 318. The reference frame memory 318 may be connected to theinter-predictor 306 to be used as the reference image against which thefuture image 300 is compared in inter-prediction operations.

The operation of the pixel predictor 302 may be configured to carry outany known pixel prediction algorithm known in the art.

The pixel predictor 302 may also comprise a filter 385 to filter thepredicted values before outputting them from the pixel predictor 302.

The operation of the prediction error encoder 303 and prediction errordecoder 304 will be described hereafter in further detail. In thefollowing examples the encoder generates images in terms of 16×16 pixelmacroblocks which go to form the full image or picture. Thus, for thefollowing examples the pixel predictor 302 outputs a series of predictedmacroblocks of size 16×16 pixels and the first summing device 321outputs a series of 16×16 pixel residual data macroblocks which mayrepresent the difference between a first macro-block in the image 300against a predicted macro-block (output of pixel predictor 302). Itwould be appreciated that other size macro blocks may be used.

The prediction error encoder 303 comprises a transform block 342 and aquantizer 344. The transform block 342 transforms the first predictionerror signal 320 to a transform domain. The transform is, for example,the DCT transform. The quantizer 344 quantizes the transform domainsignal, e.g. the DCT coefficients, to form quantized coefficients.

The entropy encoder 330 receives the output of the prediction errorencoder and may perform a suitable entropy encoding/variable lengthencoding on the signal to provide error detection and correctioncapability. Any suitable entropy encoding algorithm may be employed.

The prediction error decoder 304 receives the output from the predictionerror encoder 303 and performs the opposite processes of the predictionerror encoder 303 to produce a decoded prediction error signal 338 whichwhen combined with the prediction representation of the image block 312at the second summing device 339 produces the preliminary reconstructedimage 314. The prediction error decoder may be considered to comprise adequantizer 346, which dequantizes the quantized coefficient values,e.g. DCT coefficients, to reconstruct the transform signal and aninverse transformation block 348, which performs the inversetransformation to the reconstructed transform signal wherein the outputof the inverse transformation block 348 contains reconstructed block(s).The prediction error decoder may also comprise a macroblock filter (notshown) which may filter the reconstructed macroblock according tofurther decoded information and filter parameters.

The operation and implementation of the mode selector 310 is shown infurther detail with respect to FIG. 5. On the basis of the predictionsignals from the output of the inter-predictor 306, the output of theintra-predictor 308 and/or the image signal 300 the block processor 381determines which encoding mode to use to encode the current image block.This selection is depicted as the block 500 in FIG. 5. The blockprocessor 381 may calculate a rate-distortion cost (RD) value or anothercost value for the prediction signals which are input to the modeselector 310 and select such an encoding mode 503, 504 for which thedetermined cost is the smallest.

The mode selector 310 provides an indication of the encoding mode of thecurrent block (501). The indication may be encoded and inserted to a bitstream or stored into a memory together with the image information.

If the intra-prediction mode is selected, the block is predicted by anintra-prediction method (503). Respectively, if the inter-predictionmode is selected, the block is predicted by an inter-prediction method(504-510).

An example of the operation of the mode selector when theinter-prediction mode is selected and the type of the block is abi-predicted block, is illustrated as a block diagram in FIG. 11. Motionvector information provided by the motion vector definer 361 containsindication of a first reference block and a second reference block. Inmulti-prediction applications the motion vector information may containindication of more than two reference blocks. The block processor 381uses the motion vector information to determine which block is used as afirst reference block for the current block and which block is used as asecond reference block for the current block. The block processor 381then uses some pixel values of the first reference block to obtain firstprediction values and some pixel values of the second reference block toobtain second prediction values. For example, if a first motion vectorpoints to a fraction of a pixel (a subpixel) illustrated by the square bin the example of FIG. 12, the block processor 381 may use pixel valuesof several full pixels on the same row, for example, than said fractionof the pixel to obtain a reference pixel value. The block processor 381may use e.g. a P-tap filter such as a six-tap filter in which P pixelvalues of the reference block are used to calculate the predictionvalue. In the example of FIG. 12 these pixel values could be pixels E,F, G, H, I and J. The taps of the filter may be e.g. integer values. Anexample of such a six-tap filter is [1 −5 20 20 −5 1]/32. Hence, thefilter 1102 would receive 1101 the pixel values of pixels E, F, G, H, Iand J and filter these values by the equationP1=(E₁−5*F₁+20*G₁+20*H₁−5*I₁+J₁), in which E₁ is the value of the pixelE in the first reference block, F₁ is the value of the pixel F in thefirst reference block, G₁ is the value of the pixel G in the firstreference block, H₁ is the value of the pixel H in the first referenceblock, I₁ is the value of the pixel I in the first reference block, andJ₁ is the value of the pixel J in the first reference block. In thefirst rounding offset insertion block 1103 a first rounding offset maybe added to the value P1 i.e. P1+ rounding offset. Then, the sum may beshifted by the first shifting block 1104 to the right so that theprecision of the sum becomes M bits. The precision M is higher than theprecision of the expected prediction value. For example, pixel valuesand the prediction values may be represented by N bits wherein M>N. Insome example implementations N is 8 bits and M is 16 bits but it isobvious that also other bit lengths can be used with the presentinvention.

The second prediction can be obtained similarly by the second filter1106, which receives 1105 some pixel values of the second referenceblock. These pixel values are determined on the basis of the secondmotion vector. The second motion vector may point to the same pixel (ora fraction of the pixel) in the second reference block to which thefirst motion vector points in the first reference block (using theexample above that pixel is the subpixel b) or to another full pixel ora subpixel in the second reference block. The second filter 1106 usessimilar filter than the first filter 1102 and outputs the secondfiltering result P2. According to the example above the filter is asix-tap filter [1 −5 20 20 −5 1]/32, whereinP2=(E₂−5*F₂+20*G₂+20*H₂−5*I₂+J₂), in which E₂ is the value of the pixelE in the second reference block, F₂ is the value of the pixel F in thesecond reference block, G₂ is the value of the pixel G in the secondreference block, H₂ is the value of the pixel H in the second referenceblock, I₂ is the value of the pixel Tin the second reference block, andJ₂ is the value of the pixel J in the second reference block. In thesecond rounding offset insertion block 1107 the first rounding offsetmay be added to the value P2 i.e. P2+ rounding offset. Then, the sum maybe shifted by the second shifting block 1108 to the right so that theprecision of the sum becomes M bits.

In the combining block 1109 the two prediction values P1, P2 arecombined e.g. by summing and the combined value is added with a secondrounding value in the third rounding value insertion block 1110. Theresult is converted to a smaller precision e.g. by shifting bits of theresult to the right y times in the third shifting block 1111. Thiscorresponds with dividing the result by 2^(y). After the conversion theprecision of the prediction signal corresponds with the precision of theinput pixel values. However, the intermediate results are at a higherprecision, wherein possible rounding errors have a smaller effect to theprediction signal compared to existing methods such as the methodillustrated in FIG. 10.

In an alternative embodiment the rounding offset is not added separatelyto the results of the first 1102 and the second filter 1106 but aftercombining the results in the combining block 1110. In this case thevalue of the rounding offset is twice the value of the first roundingoffset because in the embodiment of FIG. 11 the first rounding offset isactually added twice, once to P1 and once to P2.

In some embodiments also the first shifting block 1105 and the secondshifting block 1109 are not needed when the precision of registers whichstore the filtering results is sufficient without reducing the precisionof the filtering results. In that case the third shifting block may needto shift the prediction result more than y bits to the right so that theright shifted value P has the same prediction than the input pixelvalues, for example 8 bits.

In some other example embodiments may partly differ from the above. Forexample, if a motion vector of one of the prediction directions point toan integer sample, the bit-depth of prediction samples with integeraccuracy may be increased by shifting the samples to the left so thatthe filtering can be performed with values having the same precision.

Samples of each one of the prediction directions could be rounded at anintermediate step to a bit-depth that is still larger than the inputbit-depth to make sure all the intermediate values fit to registers ofcertain length, e.g. 16-bit registers. For example, let's consider thesame example above but using filter taps: {3, −17, 78, 78, −17, 3}. ThenP1 and P2 are obtained as:P1=(3*E ₁−17*F ₁+78*G ₁+78*H ₁−17*I ₁+3*J ₁+1)>>1P2=(3*E ₂−17*F ₂+78*G ₂+78*H ₂−17*I ₂+3*J ₂+1)>>1

The bi-directional prediction signal may then be obtained using:P=(P1+P2+32)>>6.

When a motion vector points between two full pixels i.e. to a fractionof the pixel, the value for that the reference pixel value may beobtained in several ways. Some possibilities were disclosed above but inthe following some further non-limiting examples shall be provided withreference to FIG. 12.

If a motion vector points to the block labeled j the correspondingreference pixel value could be obtained by using full pixel values onthe same diagonal than j, or by a two-phase process in which e.g. pixelvalues of rows around the block j are used to calculate a set ofintermediate results and then these intermediate results could befiltered to obtain the reference pixel value. In an example embodimentthe full pixel values A and B could be used to calculate a firstintermediate result to represent a fraction pixel value aa, full pixelvalues C and D could be used to calculate a second intermediate resultto represent a fraction pixel value bb, and full pixel values E to Jcould be used to calculate a third intermediate result to represent afraction pixel value b. Similarly, fourth, fifth and sixth intermediatevalues to represent fraction pixel values s, gg, hh could be calculatedon the basis of full pixel values K to Q; R, S; and T, U. Theseintermediate results could then be filtered by a six-tap filter, forexample.

The prediction signal P obtained by the above described operations neednot be provided to a decoder but the encoder uses this information toobtain predicted blocks and prediction error. The prediction error maybe provided to the decoder so that the decoder can use correspondingoperations to obtain the predicted blocks by prediction and correct theprediction results on the basis of the prediction error. The encoder mayalso provide motion vector information to the decoder.

In an example embodiment, as is depicted in FIG. 9, the bit stream of animage comprises an indication of the beginning of an image 910, imageinformation of each block of the image 920, and indication of the end ofthe image 930. The image information of each block of the image 920 mayinclude a block type indicator 932, and motion vector information 933.It is obvious that the bit stream may also comprise other information.Further, this is only a simplified image of the bit stream and inpractical implementations the contents of the bit stream may bedifferent from what is depicted in FIG. 9.

The bit stream may further be encoded by the entropy encoder 330.

Although the embodiments above have been described with respect to thesize of the macroblock being 16×16 pixels, it would be appreciated thatthe methods and apparatus described may be configured to handlemacroblocks of different pixel sizes.

In the following the operation of an example embodiment of the decoder600 is depicted in more detail with reference to FIG. 6.

At the decoder side similar operations are performed to reconstruct theimage blocks. FIG. 6 shows a block diagram of a video decoder suitablefor employing embodiments of the invention and FIG. 7 shows a flowdiagram of an example of a method in the video decoder. The decodershows an entropy decoder 600 which performs an entropy decoding on thereceived signal. The entropy decoder thus performs the inverse operationto the entropy encoder 330 of the encoder described above. The entropydecoder 600 outputs the results of the entropy decoding to a predictionerror decoder 602 and a pixel predictor 604.

The pixel predictor 604 receives the output of the entropy decoder 600.The output of the entropy decoder 600 may include an indication on theprediction mode used in encoding the current block. A predictor selector614 within the pixel predictor 604 determines that an intra-prediction,an inter-prediction, or interpolation operation is to be carried out.The predictor selector may furthermore output a predicted representationof an image block 616 to a first combiner 613. The predictedrepresentation of the image block 616 is used in conjunction with thereconstructed prediction error signal 612 to generate a preliminaryreconstructed image 618. The preliminary reconstructed image 618 may beused in the predictor 614 or may be passed to a filter 620. The filter620 applies a filtering which outputs a final reconstructed signal 622.The final reconstructed signal 622 may be stored in a reference framememory 624, the reference frame memory 624 further being connected tothe predictor 614 for prediction operations.

The prediction error decoder 602 receives the output of the entropydecoder 600. A dequantizer 692 of the prediction error decoder 602 maydequantize the output of the entropy decoder 600 and the inversetransform block 693 may perform an inverse transform operation to thedequantized signal output by the dequantizer 692. The output of theentropy decoder 600 may also indicate that prediction error signal isnot to be applied and in this case the prediction error decoder producesan all zero output signal.

The decoder selects the 16×16 pixel residual macroblock to reconstruct.The selection of the 16×16 pixel residual macroblock to be reconstructedis shown in step 700.

The decoder receives information on the encoding mode used when thecurrent block has been encoded. The indication is decoded, whennecessary, and provided to the reconstruction processor 691 of theprediction selector 614. The reconstruction processor 691 examines theindication (block 701 in FIG. 7) and selects one of the intra-predictionmodes (block 703), if the indication indicates that the block has beenencoded using intra-prediction, or an inter-prediction mode (blocks704-711), if the indication indicates that the block has been encodedusing inter-prediction.

If the current block has been encoded using inter-prediction, the pixelpredictor 604 may operate as follows. The pixel predictor 604 receivesmotion vector information (block 704). The pixel predictor 604 alsoreceives (block 705) block type information and examines whether theblock is a bi-predicted block or not (block 706). If the block type is abi-predicted block, the pixel predictor 604 examines the motion vectorinformation to determine which reference frames and reference block inthe reference frames have been used in the construction of the motionvector information. The reconstruction processor 691 calculates themotion vectors (709) and uses the value of the (fraction of the) pixelof the reference blocks to which the motion vectors point to obtain amotion compensated prediction (710) and combines the prediction errorwith the value to obtain a reconstructed value of a pixel of the currentblock (block 711).

If the block type is a uni-predicted block, the pixel predictor 604examines the motion vector information to determine which referenceframe and reference block in the reference frame has been used in theconstruction of the motion vector information. The reconstructionprocessor 691 calculates the motion vector (707) and uses the value ofthe (fraction of the) pixel of the reference block to which the motionvector points to obtain a motion compensated prediction (708) andcombines the prediction error with the value to obtain a reconstructedvalue of a pixel of the current block (block 711).

When the motion vector does not point to a full pixel sample in thereference block, the reconstruction processor 691 calculates using e.g.a one-directional interpolation or P-tap filtering (e.g. six-tapfiltering) to obtain the values of the fractional pixels. Basically, theoperations may be performed in the same way than in the encoder i.e.maintaining the higher accuracy values during the filtering until in thefinal rounding operation the accuracy may be decreased to the accuracyof the input pixels. Therefore, the effect of possible rounding errorsmay not be so large to the predicted values than in known methods.

The above described procedures may be repeated to each pixel of thecurrent block to obtain all reconstructed pixel values for the currentblock.

In some embodiments the reconstruction processor 691 use theinterpolator 694 to perform the calculation of the fractional pixelvalues.

In some embodiments the reconstruction processor 691 provides thefractional pixel values to the predictor 695 which combines thefractional pixel values with prediction error to obtain thereconstructed values of the pixels of the current block.

In some embodiments the interpolation may also be performed by usingfull pixel values, half pixel values, and/or quarter pixel values whichmay have been stored into a reference frame memory. For example, theencoder or the decoder may comprise a reference frame memory in whichthe full pixel samples, half pixel values and quarter pixel values canbe stored.

Furthermore, in some embodiments the type of the block may also be amulti-predicted block wherein the prediction of a block may be based onmore than two reference blocks.

The embodiments of the invention described above describe the codec interms of separate encoder and decoder apparatus in order to assist theunderstanding of the processes involved. However, it would beappreciated that the apparatus, structures and operations may beimplemented as a single encoder-decoder apparatus/structure/operation.Furthermore in some embodiments of the invention the coder and decodermay share some or all common elements.

Although the above examples describe embodiments of the inventionoperating within a codec within an electronic device, it would beappreciated that the invention as described below may be implemented aspart of any video codec. Thus, for example, embodiments of the inventionmay be implemented in a video codec which may implement video codingover fixed or wired communication paths.

Thus, user equipment may comprise a video codec such as those describedin embodiments of the invention above.

It shall be appreciated that the term user equipment is intended tocover any suitable type of wireless user equipment, such as mobiletelephones, portable data processing devices or portable web browsers.

Furthermore elements of a public land mobile network (PLMN) may alsocomprise video codecs as described above.

In general, the various embodiments of the invention may be implementedin hardware or special purpose circuits, software, logic or anycombination thereof. For example, some aspects may be implemented inhardware, while other aspects may be implemented in firmware or softwarewhich may be executed by a controller, microprocessor or other computingdevice, although the invention is not limited thereto. While variousaspects of the invention may be illustrated and described as blockdiagrams, flow charts, or using some other pictorial representation, itis well understood that these blocks, apparatus, systems, techniques ormethods described herein may be implemented in, as non-limitingexamples, hardware, software, firmware, special purpose circuits orlogic, general purpose hardware or controller or other computingdevices, or some combination thereof.

The embodiments of this invention may be implemented by computersoftware executable by a data processor of the mobile device, such as inthe processor entity, or by hardware, or by a combination of softwareand hardware. Further in this regard it should be noted that any blocksof the logic flow as in the Figures may represent program steps, orinterconnected logic circuits, blocks and functions, or a combination ofprogram steps and logic circuits, blocks and functions. The software maybe stored on such physical media as memory chips, or memory blocksimplemented within the processor, magnetic media such as hard disk orfloppy disks, and optical media such as for example DVD and the datavariants thereof, CD.

The memory may be of any type suitable to the local technicalenvironment and may be implemented using any suitable data storagetechnology, such as semiconductor-based memory devices, magnetic memorydevices and systems, optical memory devices and systems, fixed memoryand removable memory. The data processors may be of any type suitable tothe local technical environment, and may include one or more of generalpurpose computers, special purpose computers, microprocessors, digitalsignal processors (DSPs) and processors based on multi-core processorarchitecture, as non-limiting examples.

Embodiments of the inventions may be practiced in various componentssuch as integrated circuit modules. The design of integrated circuits isby and large a highly automated process. Complex and powerful softwaretools are available for converting a logic level design into asemiconductor circuit design ready to be etched and formed on asemiconductor substrate.

Programs, such as those provided by Synopsys, Inc. of Mountain View,Calif. and Cadence Design, of San Jose, Calif. automatically routeconductors and locate components on a semiconductor chip using wellestablished rules of design as well as libraries of pre-stored designmodules. Once the design for a semiconductor circuit has been completed,the resultant design, in a standardized electronic format (e.g., Opus,GDSII, or the like) may be transmitted to a semiconductor fabricationfacility or “fab” for fabrication.

The foregoing description has provided by way of exemplary andnon-limiting examples a full and informative description of theexemplary embodiment of this invention. However, various modificationsand adaptations may become apparent to those skilled in the relevantarts in view of the foregoing description, when read in conjunction withthe accompanying drawings and the appended claims. However, all such andsimilar modifications of the teachings of this invention will still fallwithin the scope of this invention.

A method according to a first embodiment comprises:

determining a block of pixels of a video representation encoded in abitstream, values of said pixels having a first precision;

determining a type of the block;

if the determining indicates that the block is a block predicted byusing two or more reference blocks,

determining a first reference pixel location in a first reference blockand a second reference pixel location in a second reference block;

using said first reference pixel location to obtain a first prediction,said first prediction having a second precision, which is higher thansaid first precision;

using said second reference pixel location to obtain a secondprediction, said second prediction having the second precision, which ishigher than said first precision;

combining said first prediction and said second prediction to obtain acombined prediction; and

decreasing the precision of said combined prediction to said firstprecision.

In some methods according to the first embodiment a first roundingoffset is inserted to said first prediction and said second prediction.

In some methods according to the first embodiment the precision of saidfirst prediction and said second prediction is reduced to anintermediate prediction after adding said first rounding offset, saidintermediate prediction being higher than said first precision.

In some methods according to the first embodiment a second roundingoffset is inserted to the combined prediction before said decreasing.

In some methods according to the first embodiment said type of the blockis a bi-directional block.

In some methods according to the first embodiment said type of the blockis a multidirectional block.

In some methods according to the first embodiment the first roundingoffset is 2^(y), and said decreasing comprises right shifting thecombined prediction y+1 bits.

In some methods according to the first embodiment the first precision is8 bits.

In some methods according to the first embodiment the value of y is 5.

In some methods according to the first embodiment said first predictionand said second prediction are obtained by filtering pixel values ofsaid reference blocks.

In some methods according to the first embodiment the filtering isperformed by a P-tap filter.

An apparatus according to a second embodiment comprises:

a processor; and

a memory unit operatively connected to the processor and including:

computer code configured to determine a block of pixels of a videorepresentation encoded in a bitstream, values of said pixels having afirst precision;

computer code configured to determine a type of the block;

computer code configured to, if the determining indicates that the blockis a block predicted by using two or more reference blocks,

determine a first reference pixel location in a first reference blockand a second reference pixel location in a second reference block;

use said first reference pixel location to obtain a first prediction,said first prediction having a second precision, which is higher thansaid first precision;

use said second reference pixel location to obtain a second prediction,said second prediction having the second precision, which is higher thansaid first precision;

combine said first prediction and said second prediction to obtain acombined prediction; and

decrease the precision of said combined prediction to said firstprecision.

In some apparatuses according to the second embodiment the computer codeis further configured to insert a first rounding offset to said firstprediction and said second prediction.

In some apparatuses according to the second embodiment the computer codeis further configured to reduce the precision of said first predictionand said second prediction to an intermediate prediction after addingsaid first rounding offset, said intermediate prediction being higherthan said first precision.

In some apparatuses according to the second embodiment the computer codeis further configured to insert a second rounding offset to the combinedprediction before said decreasing.

In some apparatuses according to the second embodiment said type of theblock is a bi-directional block.

In some apparatuses according to the second embodiment said type of theblock is a multidirectional block.

In some apparatuses according to the second embodiment the firstrounding offset is 2^(y), and said decreasing comprises right shiftingthe combined prediction y+1 bits.

In some apparatuses according to the second embodiment the firstprecision is 8 bits.

In some apparatuses according to the second embodiment the value of y is5.

In some apparatuses according to the second embodiment the computer codeis further configured to obtain said first prediction and said secondprediction by filtering pixel values of said reference blocks.

In some apparatuses according to the second embodiment said filteringcomprises a P-tap filter.

According to a third embodiment there is provided a computer readablestorage medium stored with code thereon for use by an apparatus, whichwhen executed by a processor, causes the apparatus to:

determine a block of pixels of a video representation encoded in abitstream, values of said pixels having a first precision;

determine a type of the block;

if the determining indicates that the block is a block predicted byusing two or more reference blocks,

determine a first reference pixel location in a first reference blockand a second reference pixel location in a second reference block;

use said first reference pixel location to obtain a first prediction,said first prediction having a second precision, which is higher thansaid first precision;

use said second reference pixel location to obtain a second prediction,said second prediction having the second precision, which is higher thansaid first precision;

combine said first prediction and said second prediction to obtain acombined prediction; and

decrease the precision of said combined prediction to said firstprecision.

According to a fourth embodiment there is provided at least oneprocessor and at least one memory, said at least one memory stored withcode thereon, which when executed by said at least one processor, causesan apparatus to perform:

determine a block of pixels of a video representation encoded in abitstream, values of said pixels having a first precision;

determine a type of the block;

if the determining indicates that the block is a block predicted byusing two or more reference blocks,

determine a first reference pixel location in a first reference blockand a second reference pixel location in a second reference block;

use said first reference pixel location to obtain a first prediction,said first prediction having a second precision, which is higher thansaid first precision;

use said second reference pixel location to obtain a second prediction,said second prediction having the second precision, which is higher thansaid first precision;

combine said first prediction and said second prediction to obtain acombined prediction; and

decrease the precision of said combined prediction to said firstprecision.

According to some example embodiments the apparatus is an encoder.

According to some example embodiments the apparatus is a decoder.

An apparatus according to a fifth embodiment comprises:

an input to determine a block of pixels of a video representationencoded in a bitstream, values of said pixels having a first precision;

a determinator to determine a type of the block; wherein if thedetermining indicates that the block is a block predicted by using twoor more reference blocks, said determinator further to determine a firstreference pixel location in a first reference block and a secondreference pixel location in a second reference block;

a first predictor to use said first reference pixel location to obtain afirst prediction, said first prediction having a second precision, whichis higher than said first precision;

a second predictor to use said second reference pixel location to obtaina second prediction, said second prediction having the second precision,which is higher than said first precision;

a combiner to combine said first prediction and said second predictionto obtain a combined prediction; and

a shifter to decrease the precision of said combined prediction to saidfirst precision.

An apparatus according to a sixth embodiment comprises:

means for determining a block of pixels of a video representationencoded in a bitstream, values of said pixels having a first precision;

means for determining a type of the block;

means for determining a first reference pixel location in a firstreference block and a second reference pixel location in a secondreference block, if the determining indicates that the block is a blockpredicted by using two or more reference blocks;

means for using said first reference pixel location to obtain a firstprediction, said first prediction having a second precision, which ishigher than said first precision;

means for using said second reference pixel location to obtain a secondprediction, said second prediction having the second precision, which ishigher than said first precision;

means for combining said first prediction and said second prediction toobtain a combined prediction; and

means for decreasing the precision of said combined prediction to saidfirst precision.

What is claimed is:
 1. A method for decoding or encoding video, themethod comprising: determining a coding type of a block of pixels of thevideo, values of said pixels having a first precision, wherein the firstprecision indicates the number of bits needed to represent values ofsaid pixels; if the determining indicates that the block is a blockpredicted by using two or more reference blocks, the reference blockshaving said first precision: determining a first reference pixellocation in a first reference block and a second reference pixellocation in a second reference block; using said first reference pixellocation to obtain a first prediction by a fractional pixel sampleinterpolation process, said first prediction having a second precision,which is higher than said first precision; using said second referencepixel location to obtain a second prediction, said second predictionhaving the second precision, which is higher than said first precision,wherein the second precision indicates the number of bits needed torepresent values of said first prediction and values of said secondprediction; adding said first prediction having the second precision andsaid second prediction having the second precision and a rounding valueto obtain a combined prediction; and after adding the first predictionand the second prediction and the rounding value, decreasing theprecision of said combined prediction to said first precision byshifting bits of the combined prediction to the right.
 2. The methodaccording to claim 1, wherein said second reference pixel location is aninteger sample, and wherein said second prediction is obtained byshifting value of said second reference pixel location to the left. 3.The method according to claim 1 further comprising: inserting a firstrounding offset to said first prediction and said second prediction. 4.The method according to claim 3 further comprising: inserting a secondrounding offset to the combined prediction before said decreasing. 5.The method according to claim 3, wherein the first rounding offset is2^(y), and said decreasing comprises right shifting the combinedprediction y+1 bit.
 6. The method according to claim 1 furthercomprising: reducing the precision of said first prediction and saidsecond prediction to an intermediate prediction after adding a firstrounding offset, said intermediate prediction being higher than saidfirst precision.
 7. The method according to claim 1, wherein said typeof the block is a bi-directional block or a multidirectional block. 8.An apparatus for decoding or encoding video, the apparatus comprising:at least one processor and at least one memory including computerprogram code, the at least one memory and computer program codeconfigured to, with the processor, cause the apparatus to: determine anencoding type of a block of pixels of a video, values of said pixelshaving a first precision, wherein the first precision indicates thenumber of bits needed to represent values of said pixels; wherein if thedetermining indicates that the block is a block predicted by using twoor more reference blocks, the reference blocks having said firstprecision: determine a first reference pixel location in a firstreference block and a second reference pixel location in a secondreference block; use said first reference pixel location to obtain afirst prediction by a fractional pixel sample interpolation process,said first prediction having a second precision, which is higher thansaid first precision; use said second reference pixel location to obtaina second prediction, said second prediction having the second precision,which is higher than said first precision, wherein the second precisionindicates the number of bits needed to represent values of said firstprediction and values of said second prediction; add said firstprediction having the second precision and said second prediction havingthe second precision and a rounding value to obtain a combinedprediction; and after adding the first prediction and the secondprediction and the rounding value, decrease the precision of saidcombined prediction to said first precision by shifting bits of thecombined prediction to the right.
 9. The apparatus according to claim 8,wherein said second reference pixel location is an integer sample, andwherein said second prediction is obtained by shifting value of saidsecond reference pixel location to the left.
 10. The apparatus accordingto claim 8, wherein the at least one memory and computer code arefurther configured to: insert a first rounding offset to said firstprediction and said second prediction.
 11. The apparatus according toclaim 10, wherein the at least one memory and computer code are furtherconfigured to: insert a second rounding offset to the combinedprediction before said decreasing.
 12. The apparatus according to claim10, wherein the first rounding offset is 2^(y), and said decreasingcomprises right shifting the combined prediction y+1 bits.
 13. Theapparatus according to claim 8, wherein the at least one memory andcomputer code are further configured to: reduce the precision of saidfirst prediction and said second prediction to an intermediateprediction after adding said first rounding offset, said intermediateprediction being higher than said first precision.
 14. The apparatusaccording to claim 8, wherein said type of the block is a bi-directionalblock or a multidirectional block.
 15. A computer program product fordecoding or encoding video, the computer program product comprising atleast one non-transitory computer readable storage medium havingcomputer executable program code portions stored therein, the computerexecutable program code portions comprising program code instructionsconfigured to: determine a coding type of a block of pixels of thevideo, values of said pixels having a first precision, wherein the firstprecision indicates the number of bits needed to represent values ofsaid pixels; if the determining indicates that the block is a blockpredicted by using two or more reference blocks, the reference blockshaving said first precision: determine a first reference pixel locationin a first reference block and a second reference pixel location in asecond reference block; use said first reference pixel location toobtain a first prediction by a fractional pixel sample interpolationprocess, said first prediction having a second precision, which ishigher than said first precision; use said second reference pixellocation to obtain a second prediction, said second prediction havingthe second precision, which is higher than said first precision, whereinthe second precision indicates the number of bits needed to representvalues of said first prediction and values of said second prediction;add said first prediction having the second precision and said secondprediction having the second precision and a rounding value to obtain acombined prediction; and after adding the first prediction and thesecond prediction and the rounding value, decrease the precision of saidcombined prediction to said first precision by shifting bits of thecombined prediction to the right.
 16. The computer program productaccording to claim 15, wherein said second reference pixel location isan integer sample, and wherein said second prediction is obtained byshifting value of said second reference pixel location to the left. 17.The computer program product according to claim 15 further comprising:inserting a first rounding offset to said first prediction and saidsecond prediction; and inserting a second rounding offset to thecombined prediction before said decreasing.
 18. The computer programproduct according to claim 15, further comprising inserting a firstrounding offset to said first prediction and said second prediction,wherein the first rounding offset is 2^(y), and said decreasingcomprises right shifting the combined prediction y+1 bit.
 19. Anapparatus for decoding or encoding video, the apparatus comprising: adeterminator to determine a coding type of a block of pixels of thevideo, values of said pixels having a first precision, wherein the firstprecision indicates the number of bits needed to represent values ofsaid pixels; wherein if the determining indicates that the block is ablock predicted by using two or more reference blocks with the referenceblocks having said first precision, said determinator further todetermine a first reference pixel location in a first reference blockand a second reference pixel location in a second reference block; afirst predictor to use said first reference pixel location to obtain afirst prediction by a fractional pixel sample interpolation process,said first prediction having a second precision, which is higher thansaid first precision; a second predictor to use said second referencepixel location to obtain a second prediction, said second predictionhaving the second precision, which is higher than said first precision,wherein the second precision indicates the number of bits needed torepresent values of said first prediction and values of said secondprediction; a combiner to add said first prediction having the secondprecision and said second prediction having the second precision and arounding value to obtain a combined prediction; and after adding thefirst prediction and the second prediction and the rounding value, ashifter to decrease the precision of said combined prediction to saidfirst precision by shifting bits of the combined prediction to theright.
 20. The apparatus according to claim 19, wherein said secondreference pixel location is an integer sample, and wherein said secondprediction is obtained by shifting value of said second reference pixellocation to the left.
 21. The apparatus according to claim 19 furtherconfigured to: insert a first rounding offset to said first predictionand said second prediction; and insert a second rounding offset to thecombined prediction before said decreasing.
 22. The apparatus accordingto claim 19, further configured to insert a first rounding offset tosaid first prediction and said second prediction, wherein the firstrounding offset is 2^(y), and said decreasing comprises right shiftingthe combined prediction y+1 bit.
 23. An apparatus for decoding orencoding video, the apparatus comprising: means for determining a codingtype of a block of pixels of the video, values of said pixels having afirst precision, wherein the first precision indicates the number ofbits needed to represent values of said pixels; means for determining afirst reference pixel location in a first reference block and a secondreference pixel location in a second reference block, if the determiningindicates that the block is a block predicted by using two or morereference blocks with the reference blocks having said first precision;means for using said first reference pixel location to obtain a firstprediction by a fractional pixel sample interpolation process, saidfirst prediction having a second precision, which is higher than saidfirst precision; means for using said second reference pixel location toobtain a second prediction, said second prediction having the secondprecision, which is higher than said first precision, wherein the secondprecision indicates the number of bits needed to represent values ofsaid first prediction and values of said second prediction; means foradding said first prediction having the second precision and said secondprediction having the second precision and a rounding value to obtain acombined prediction; and after adding the first prediction and thesecond prediction and the rounding value, means for decreasing theprecision of said combined prediction to said first precision byshifting bits of the combined prediction to the right.
 24. The apparatusaccording to claim 23, wherein said second reference pixel location isan integer sample, and wherein said second prediction is obtained byshifting value of said second reference pixel location to the left. 25.The apparatus according to claim 23 further comprising: means forinserting a first rounding offset to said first prediction and saidsecond prediction; and means for inserting a second rounding offset tothe combined prediction before said decreasing.
 26. The apparatusaccording to claim 23, further comprising means for inserting a firstrounding offset to said first prediction and said second prediction,wherein the first rounding offset is 2^(y), and said decreasingcomprises right shifting the combined prediction y+1 bit.